---
title: Manage credentials (API keys)
description: Learn how to manage credentials (API keys) in TensorZero.
---

This guide explains how to manage credentials (API keys) in TensorZero Gateway.

Typically, the TensorZero Gateway will look for credentials like API keys using standard environment variables.
The gateway will load credentials from the environment variables on startup, and your application doesn't need to have access to the credentials.

That said, you can customize this behavior by setting alternative credential locations for each provider.
For example, you can provide credentials dynamically at inference time, or set alternative static credentials for each provider (e.g. to use multiple API keys for the same provider).

## Default Behavior

By default, the TensorZero Gateway will look for credentials in the following environment variables:

| Model Provider                                                                      | Default Credential            |
| ----------------------------------------------------------------------------------- | ----------------------------- |
| [Anthropic](/integrations/model-providers/anthropic/)                               | `ANTHROPIC_API_KEY`           |
| [AWS Bedrock](/integrations/model-providers/aws-bedrock/)                           | Uses AWS SDK credentials      |
| [AWS SageMaker](/integrations/model-providers/aws-sagemaker/)                       | Uses AWS SDK credentials      |
| [Azure](/integrations/model-providers/azure/)                                       | `AZURE_OPENAI_API_KEY`        |
| [Deepseek](/integrations/model-providers/deepseek/)                                 | `DEEPSEEK_API_KEY`            |
| [Fireworks](/integrations/model-providers/fireworks/)                               | `FIREWORKS_API_KEY`           |
| [GCP Vertex AI (Anthropic)](/integrations/model-providers/gcp-vertex-ai-anthropic/) | `GCP_VERTEX_CREDENTIALS_PATH` |
| [GCP Vertex AI (Gemini)](/integrations/model-providers/gcp-vertex-ai-gemini/)       | `GCP_VERTEX_CREDENTIALS_PATH` |
| [Google AI Studio (Gemini)](/integrations/model-providers/google-ai-studio-gemini/) | `GOOGLE_API_KEY`              |
| [Groq](/integrations/model-providers/groq/)                                         | `GROQ_API_KEY`                |
| [Hyperbolic](/integrations/model-providers/hyperbolic/)                             | `HYPERBOLIC_API_KEY`          |
| [Mistral](/integrations/model-providers/mistral/)                                   | `MISTRAL_API_KEY`             |
| [OpenAI](/integrations/model-providers/openai/)                                     | `OPENAI_API_KEY`              |
| [OpenAI-Compatible](/integrations/model-providers/openai-compatible/)               | `OPENAI_API_KEY`              |
| [OpenRouter](/integrations/model-providers/openrouter/)                             | `OPENROUTER_API_KEY`          |
| [SGLang](/integrations/model-providers/sglang/)                                     | `SGLANG_API_KEY`              |
| [Text Generation Inference (TGI)](/integrations/model-providers/tgi/)               | None                          |
| [Together](/integrations/model-providers/together/)                                 | `TOGETHER_API_KEY`            |
| [vLLM](/integrations/model-providers/vllm/)                                         | None                          |
| [XAI](/integrations/model-providers/xai/)                                           | `XAI_API_KEY`                 |

## Customizing Credential Management

You can customize the source of credentials for each provider.

See [Configuration Reference](/gateway/configuration-reference/) (e.g. `api_key_location`) for more information on the different ways to configure credentials for each provider.
Also see the relevant provider guides for more information on how to configure credentials for each provider.

### Static Credentials

You can set alternative static credentials for each provider.

For example, let's say we want to use a different environment variable for an OpenAI provider.
We can customize variable name by setting the `api_key_location` to `env::MY_OTHER_OPENAI_API_KEY`.

```toml
[models.gpt_4o_mini.providers.my_other_openai]
type = "openai"
api_key_location = "env::MY_OTHER_OPENAI_API_KEY"
# ...
```

At startup, the TensorZero Gateway will look for the `MY_OTHER_OPENAI_API_KEY` environment variable and use that value for the API key.

<Tip>

#### Load Balancing Between Multiple Credentials

You can load balance between different API keys for the same provider by defining multiple variants and models.

For example, the configuration below will split the traffic between two different OpenAI API keys, `OPENAI_API_KEY_1` and `OPENAI_API_KEY_2`.

```toml
[models.gpt_4o_mini_1]
routing = ["openai"]

[models.gpt_4o_mini_1.providers.openai]
type = "openai"
model_name = "gpt-4o-mini"
api_key_location = "env::OPENAI_API_KEY_1"

[models.gpt_4o_mini_2]
routing = ["openai"]

[models.gpt_4o_mini_2.providers.openai]
type = "openai"
model_name = "gpt-4o-mini"
api_key_location = "env::OPENAI_API_KEY_2"

[functions.generate_haiku]
type = "chat"

[functions.generate_haiku.variants.gpt_4o_mini_1]
type = "chat_completion"
model = "gpt_4o_mini_1"

[functions.generate_haiku.variants.gpt_4o_mini_2]
type = "chat_completion"
model = "gpt_4o_mini_2"
```

You can use the same principle to set up fallbacks between different API keys for the same provider.
See [Retries & Fallbacks](/gateway/guides/retries-fallbacks/) for more information on how to configure retries and fallbacks.

</Tip>

### Dynamic Credentials

You can provide API keys dynamically at inference time.

To do this, you can use the `dynamic::` prefix in the relevant credential field in the provider configuration.

For example, let's say we want to provide dynamic API keys for the OpenAI provider.

```toml {7}
[models.user_gpt_4o_mini]
routing = ["openai"]

[models.user_gpt_4o_mini.providers.openai]
type = "openai"
model_name = "gpt-4o-mini"
api_key_location = "dynamic::customer_openai_api_key"
```

At inference time, you can provide the API key in the `credentials` argument.

```python {14-16}
from tensorzero import TensorZeroGateway

with TensorZeroGateway.build_http(gateway_url="http://localhost:3000") as client:
    response = client.inference(
        function_name="generate_haiku",
        input={
            "messages": [
                {
                    "role": "user",
                    "content": "Write a haiku about artificial intelligence.",
                }
            ]
        },
        credentials={
            "customer_openai_api_key": "sk-..."
        }
    )

print(response)
```

### Configure credential fallbacks

You can configure fallback credentials that will be used automatically if the primary credential fails.

This is particularly useful for calling functions and models that require dynamic credentials from the TensorZero UI (by falling back to static credentials).

To configure a fallback, use an object with `default` and `fallback` fields instead of a simple string:

```toml {7}
[models.gpt_4o_mini]
routing = ["openai"]

[models.gpt_4o_mini.providers.openai]
type = "openai"
model_name = "gpt-4o-mini"
api_key_location = { default = "dynamic::customer_openai_api_key", fallback = "env::OPENAI_API_KEY" }
```

At inference time, the gateway will first try to use the dynamic credential.
If that fails, it will automatically fall back to the environment variable.

### Set default credentials for a provider type

Most model providers have default credential locations.
For example, OpenAI's `api_key_location` defaults to `env::OPENAI_API_KEY`.
These credentials apply to the default function and shorthand models (e.g. calling the model `openai::gpt-5`).

You can override the default location for a particular provider using `[provider_types.YOUR_PROVIDER_TYPE.defaults]`.
For example, we can override the default location for the OpenAI provider type to require a dynamic API key:

```toml title="tensorzero.toml"
[provider_types.openai.defaults]
api_key_location = "dynamic::customer_openai_api_key"
# ...
```

Unless otherwise specified, every model provider of type `openai` will require the `customer_openai_api_key` credential.

See the [Configuration Reference](/gateway/configuration-reference) for more details.
